EBN notebook Clinically useful measures of the effects of treatment

In the EBN notebooks that have appeared in the previous 2 issues of the journal, we outlined 3 steps to help us to determine whether to apply the results of a research study to our patients. Firstly, we should consider whether the study results are valid. For articles about the effectiveness of healthcare interventions, the 3 key validity issues are whether the patients were randomly assigned to different treatments, whether they were analysed according to the groups to which they were assigned, and the extent of follow up. Secondly, if we judge the study to be valid, we examine the study results to determine whether the new treatment is effective, the size of the effect, and whether the effect is clinically important. When determining the clinical significance of effective treatments, findings can be expressed in 3 ways: as a change in relative risk, change in absolute risk and number needed to treat (NNT). Abstracts in Evidence-Based Nursing that describe effective treatments include these numbers, when data permit their calculation. The third step, the application to an individual patient, requires knowledge about both the study and the patient. This involves consideration of both the extent to which the patient resembles those who were enrolled in the study and the patient’s risk for the event for which the treatment was designed. This notebook will explain the concepts that help us to determine whether study findings should be applied to our own individual patients. Let’s work through a randomised controlled trial abstracted in this issue of the journal (p52) that evaluates the effectiveness of a cognitive behavioural family intervention in reducing psychological distress and depression in caregivers of patients with Alzheimer’s disease. Addressing first the validity of this trial, we find that patient-caregiver dyads were randomly assigned to the 14 session family intervention or to 1 of 2 control groups (1 of which received a cathartic interview and 1 of which did not receive the family intervention or the interview), the patients were analysed in the groups to which they were assigned, and loss to follow up at 3 months was only 2%. Turning to the results, let’s compare the intervention group with the no interview control group. We learn that within 3 months, 23% of the caregivers who received the family intervention met criteria for psychological distress (we will call this the experimental event rate or EER), whereas 77% of those in the no interview control group met the same criteria (we will call this the control event rate or CER). This difference was statistically significant. Now that we know the results, let’s examine the application to our patients. We begin with the traditional measure of effect, which is the relative risk reduction (RRR), defined as the proportional reduction in rates of bad outcomes between experimental and control participants in a trial, calculated as (CER–EER)/ CER.* The RRR is (77%–23%)/77% or 70%; that is, the family intervention reduced the risk of caregivers experiencing psychological distress by 70%. Why not confine our description of the clinical significance of this result to the RRR? The reason is that the RRR fails to discriminate huge absolute treatment effects from those that are trivial. For example, if the rates of psychological distress were 10 times less than those observed in this trial, and only 7.7% of control group caregivers and 2.3% of treatment group caregivers experienced psychological distress, the RRR would still be 70%. This is because the RRR ignores how rarely or commonly the outcome in question occurs anyway in the patients entering the trial (known as baseline risk); as a result, these measures cannot discriminate huge benefits and risks from small ones. The effect of treating low risk caregivers will differ from the effect of treating caregivers at a much higher risk of psychological distress. In contrast to the non-discriminating RRR, the absolute differences in the rates of psychological distress between control and experimental group caregivers (ie, CER–EER) clearly do discriminate between these extremes; this measure is called the absolute risk reduction (ARR) and is defined as the absolute arithmetic difference in rates of bad outcomes between experimental and control participants in a trial.† In the caregiver trial, the ARR = CER–EER = 77%–23% = 54%. In the hypothetical example provided above, in which 2.3% of experimental group caregivers and 7.7% of control group caregivers had psychological distress, the ARR = CER–EER = 7.7%–2.3% = 5.4%. The absolute differences take into account the baseline risk of patients and provide more detailed information than RRRs. But unlike the RRRs that can be recalled as whole numbers, ARRs are decimals and therefore more difficult to remember. If however, we divide the ARR into 1 (ie, if we invert the ARR or take its reciprocal, so that it becomes 1/ARR), we generate a very useful number that represents the number of patients we need to treat (NNT) with the experimental treatment for the duration of the trial to prevent 1 additional bad outcome (eg, psychological distress). In the caregiver trial, we generate the number of caregivers we would need to treat with the family intervention to prevent 1 additional caregiver from experiencing psychological distress within 3 months of the intervention. The NNT is 1/ARR or 1/54% or 1/0.54 = 1.85; we usually round this number upward (because we can’t have part of a person!), and we can now say that for every 2 caregivers who receive the family intervention, 1 case of psychological distress will be averted. Using the hypothetical example provided above in which the ARR was 5.4% rather than 54%, the NNT is 1/ARR or 1/0.054 = 18.5 or 19 caregivers who would need to receive the family intervention to prevent 1 additional caregiver from experiencing psychological distress. The NNT is higher for patients at lower risk of the health problem.